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SYSTEM. 



fe ^alS^tr^arS^^^^^^^ T.e v.eo and aud.o s.gnals 

Of a m.ted transmission capacity. At least one of the " fimLn^^ 

detected. According to this detection, the transrrSssSn con ^nrf . '^"'^ ^'S^als .s 

rat,o of the video and audio signals and the cZctfroM^^^^^^ ''°T"^'' ^"^P"" *° '''^ transmission 
two s,gnais. Hence, the transmission qualityT^Coved esoS^^^^^ "^""^^'"^ ^P««^« °^ 

insufficient transmission capacities. 'mproved. especially ,n TV conference systems having only 
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VIDEO AND AUDIO MULTIPLEX TRANSMISSION SYSTEM 
TECHNICAL FIELD oartlcularly 

^i=^=S=BI-- 



10 



BACKGROUND ART 



or the B system which uses one 84 >*/s Hne. tne ^^^^^ gp^^,, ,s 56 ^^^^ 

. jS.^ Se vldeo^lng speed Is 1 12 kb/s). ^ 3, .,3 ,„herently different 

H^ev^the ratio of the data density between the ?herefore. in the case of a 

by hu^red-fold. Despite this, the f ^^^^ .^^^fsmC time becomes small, invtng 

» ZTT^^ ^rtiTS^ ^a'SS^Ulu.'.h. 

video signals with respect to the sound ^^•''^^^f^° However, there was a problem in that the a=tua 
kos/s worth or 56 kps/s worth of transm.ss.on capacity .s secured. 



periods in audio signals for the video signals. 
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DISCLOSURE 01= THE INVENTION „,„ da.- 
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Rrst this object Is realized as follows according to one aspect of the oresent inv«n«n„ i - 
of data of at least ana nf th- JlL P"^*"''® accordance with the amount 

single most suitable c<5ing bft rSe Is seterlS rl/ w ^^^^s. The 

audio quality doeslc^rt! butTal?^^ ^'^h the content of the transmission, so the 

quality Of me picLre ^^^ s^^^^ h transmission can be performed at the 

rscel, rarord^'ceT^'t^e^LTr^^^^^^ ^ - P^*'--' « the 

" sele^^JSm^'t;;'::^^^^^^^^ ^^^^^ -table quality, that Is. bit rate, .s 

. =5SH?J~SrJE~=— ^^^^^ 

among frSnTs ° '^^^^ '"^''^ °' the rate of change of data 

-0 data.^^,e^ef"e'°n; cTmprisfdTa' J°dr cZ '''T, °' '""'^'^'"^ 

variable length coding unTfor coin T^ia^eTennl ^ 

generating delay time data for svnrhroni,i„o. ^ P'cture. a delay amount calculation unit for 

data and audio data and the delartlme dL L a recTn^^^^^^ multiplexing the coded v.deo 
demultiplexing a multiplexed signL Z a LsmiL on C demultiplexing un.t for 

data, a variable delay control unit for delavino ti^ luH^ ' ^"'^ ''^'^^ ''""^ 

audio decoding unit 'for decoding audio d'ite romT var aWe de^al T '^'^^ "'"^ 
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25 



coding video signals and a variable length coding T"^^^^. id is comprised so that the output 
p°^:^^ed. to deS wfth audio signals, with^ ^^^^Jf^^^^^^^^^^^^^ the said audte coding un« are , 
from said variable length codmg unrt and *«^*P"^ ^ ^^.^ and audio signal from wha^ .s 

, transmitted multiplexed. Je IJ^^'^^t^ston system being provided with a time <i^-J^^-J^'^^ 
transmitted, said video and audio '""«P'«V^*';'^^^^^ gound and assembling ft into a packet based on 
St for extracting a signal during an effective 53 comprised so as to notify a system 

oXut from said audio coding unit ^^^\^^^lZ Z^^J,t is comprised so as to receive sa.d 
^1 unit of an audio transmission f P««J- ,f .^^^^ ^o^^Lg the coding amount of said video codmg 

,0 t^dio transmission speed and "^^^^^^^^S^l '„°eTorv in said variable length coding unit, and 
^^^.:^''::^^^^oT:. S^i^l ^ aU^-m,at suited for s.d audio transmis.on speed. 

IS BRIEF OESCRIFnON OF THE DRAWINQS 

. . of a video and audio multiplex transmission system: 

? " SHk diaa.»" 0. Ih. »»»mlt»r side o. ». «nbodln,»* 

c:« a chaws the audio frame format; 

Rg. 12 is an explanatory view rf the demultpteMn^^^^ 

« z 30 1 a r.^s'r LrrcT. d^^'-T^e «. „» ^ ^ - »• 

Fig. 33 shows an example of a bit array of PAS data: 
Fig. 34 shows an example of the allocation of BAS codes 

Fig. 41 is a block diagram showing an embodiment of ine oeiay a 
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shown in Rg. 39: 

to 

BEST MODE FOR CARRYfNQ OUT THE INVENTION 

« "i«ersBnd|™o(th«prss8«inv6.aoS '^'"""^ """9 "9. t to Rg. 5 so lo facilitate 

20 coding unit 2 and input to TZbX no^S fl ^° ^' ''^ an audio 

coded video data Is temporarily stored U^tluZ j,dt "^J^^^ "^^ "^^^ 

4 is comprised to have such a function The ^S^^Z L^d ^ohT^ ®- "^^ 

«. a»llo ceding JtTte ,Z!y^, S?^ T!' ' " "Sn* » » 

unit ,3 i, . o,A convarar. and .9 ia-a syS^^^itnT' " * '2 » an audio decoding 

signal la codad a. m, audio coS^ JnTj^^.T^z'^^nn'.rl' « °" tna audio 

made of a dslay conwl „„« 7 to, aM^ a d^l^ J™™ !. " ' ^ f™"'""" " 

5 and 1 1 . ^ ™ ' " "uHptolng and damulllplaxlng units B 

un,tS»Se°iZn1';^nZS:.'""" '""""'"^ »» -tarfac. 



°r " »V --^ -uniple^g .no 

Wdeo signal d«:odlng u„., 14 ^ OA convX .r^n't UH' ''""^ """"'9'' "» 

passas ,„e audb dacodlng unj 12 ana D« co'Sr^e, n a^d « l.^i"!?" f '^^ 

NO. .a. .aa^ount =, delay c..d, r ,„ .e .^^t^'X't^J'J^^Z 



SO 



where, 

55 tvi: delay time at the processing units 3. 41 , and 42 
tv2: delay time at the processing units 14 and 1 5 
tAi: delay time at the processing units 1. 2, and 7 
tAz: delay time at the processing units 12 and 13 
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corr^^ to Fig. 3. and « show, a variable "^'^ «rtabl. I.n3«' cod«i .Id~ 

data. This is read out from the DuHer momofV w 

demuWpleong ur«s 6 ard 1 1 . ^ ,„ memory 43. the «a9e Is ™a*«l 

It ir. aiconianco ««. «ie amount of video ,g jtorm in the illustraoon of Fig. 4(B). 

, «l,e . me storage amoui« SKoeeds the second .hres>»W »*ue isas video codi.^ 

^Dle Ibl systKii eonwl unit 19 "^„^, Further. «l»n the stage Is reached 

L <i«ce.o" trth^Srili^^Ter-le'itm oont,^ un« ,0 comm-s 
„he.e the storage amount «'» """if ^I^ScSon is made liner. 

^-^Tgure S.B, -he .erne ,or™. . th, ^ -.^^L" ''^SS 

^rj^:r^r^-Ss:rjJ«-rj?-r^^^^ 

«w frtr the first haif 128 kbps portion. Further, Pig. ^K^)^ 
usS i me video signal portion until 384 i*^ is reached. p^^lon^ Por 

in the conventional case, as shown m ^' processing in the video coding unit 41 s 

•-r-em"^-^ P~SLn,*r;^se« .-e^. . ccmpHsed o. . 

Figure 6 is a view of the basic constitution of me p 
« transmission unit B and a reception unit C. ^ audio multiplexing system or 

Rcmre 7 shows one embodiment of the transmitter ^-de of tt.e via ^ ^jjapa^Q coding 

t.e Xnt invention, in which embodiment ^l^^f^, 3^^^^^^^^^^ wiS, respect U, me digit^ 

units (for example ADPCMs) 2, to 2. each with ""^^^^^ ^n a noise evaluation method from the 

soSd an evaluation unit 2A for determining ^ :^f\^^^^r<S^^ sound and generates a coding brt 
35 re^oduced outputs of the adaptive codinguruts 2. '° ^ ^J*^ corresponding adaptive coding 

rarita. and a selection unit 28 for ^electng me ou put aud 0^^^^ evaluation un. 2A 

unU bt me coding bit rate data. The f^Ti ^in^^^^^^^^ '<'^ ^"^ '"'^H'^m^ 

Further, me video coding unit 4 is ^ll^'^^^ZVfsi^^O) 42 for coding in variablejengms the 

prcuire from me A.'0 converter 3- a ^-^'^^^^^^^^^ Um coded video data. Furmer. 9 Is 

- V^Z:^^^^^^^^^ It Sr-ing units 2, to 2. is as shown 

so The SsmissioS line in me frame formats sho-n address space for temporarily wnfng 

rmV^drdrTudifdr^^^^^^^^ — - ^° - '-^"^^ " ° 

r^-rSg^da^ - ^^da^ as the control data .0 (application channel. 



EP 0 418 396 A1 



4S 



SO 



ss 



data). Also, one multtframe is comprised of 16 transmission f««- 

comprised and sent as shown in Rg 2(a) Zd n^TZ n !^!f' ""^"^""^ above, but the data is" 
. frames, tar example. F,., to F, " and F tTp ^''^^^ *^^^«<^«'"» HI -> POJ in pairs of tm> transmisSon 
the transmission framed Frand F, ^ """^ '««'«^«^ ^re dlSLm°^ 

' ^r<^'^-re.^ofT^ey'!^^^ synchronization by the synchronization 

to distinguish frame units by L^oTm^cHt ^l ^^^ synchronization. That Is. it is possibli 

discrimination of ail frames is necessa^7or • differentiate frames by (2). Further 

(bit a«ocatlon signal) data Jexpl^neTfater.^^^^^^^ ""'^ "'"^ '^'P** ^ BaS 

" the «V:SeXlVmtL:'t^^^^^^^^^^ ;ntam,a«on and the amount of video data at 

evaluation unit 2A. the coding bit rate SroorlS^ t fh ^/^T"^ °' ^"^'"9 fr^"" «he 

After the establishment of uLTz^n^ZTauTn lUs^^^^ T ^ '^^^ " ^"'^ transmitted, 

data. The BAS data is judged every sub^um?^« ;onI ^1 demultiplexing of the 

'5 12. the next coding bit rati is T^Z^l^^^V^ .^^^^ " ^ «"b-multiframes). As shown in F«. 
eight frames) fram me BAS dam ISlfeforf ' frames or more out of the 

Ke mSSnTunTe ^^^^^^^^^ by the embodiment 

. r.s^a;r:r;v^^e--:™ . 
^'?^2?rrf-ro^dris%s^^^^^^^ 

units in the vertical :^Xlor i:Zo^'J':'''l''1 ^r each 8 bit 

8 bit gmup of the tower stage an^ l^^im!^°'^ ^ *° "^^^ '^^^ « ^^ift to the 
as the selected coding bit rate. " ""^ successively in the form of correspondence with 

Video' dts Tm:iiT:z::t ::r:oT.T::::^r Tr' »^ -^^^ « 

audio data. ^ P*""*'"^ ^"^^ « completion of fllfing in of the stages of 

00 audto SlTvSf d™ l'r;l,l*!:'rr "^-^ '^'^^ -'^■°-'"P ^etwe- 

coding, and 7^,lt coding. ^ transm,ss.on frame .n the case of 04,it coding, i-blt coding. 3-bit 

rate crbTcSanTe::^?^^^^^^ r^^f^^ir'"^ - the audio coding 

s ^i^'^'tr:\X:Z:^^^^^ -y. me data can be stored for every 8 

The format of the transmlss on f?^L in it ^ u ^ mentioned Rg. 23. 
BAS data is shown in^ TrS tf^S^Lr Z'^LZ ''" """"^ '^^^'^^^ - the 

horizontal bits. as iHustrared (see Rg 9,^rn Sr.L^^^^^^^^ °' "'t and 3 

Shown in Rg. ,9. each transrnission^amerfo^nT^dTor T *" """^""'^ 'li^^ction as 

' and 7-bit coding as shown in Rg. 20 t^g 23 °' '"'^ ""^^ °' ««*'"9- '"b't coding. 3-bit coding. 

transm^rtr^XroTr^^^^^^ ^''^^ ^^-^ RAM 27. When the 

matching the transmission speed transmission line 10 at a rate 

mu,.JjLrgpt%st:nrnVg^^^^^^^^^ same construction as the 

multi^am^^rnchrLtnS^^^^^^^ synchronization and r2, 

by the demultiplexing unit 1 1 based on t^e B^l da a' ^LTT" '^'"^ "^^'^ demultiplexed 

the various interfaces, with the coded bVrate data sent fo ^hl LZ ^^^^^ '''^^"^h 

"'NeTrfur j'- '^'^^ data;:rvfdT;ro;:n;t"^r"' ^^^^ '° - 

decor^aurdf^rar^^^^^^ ''^ ^- from the decoding control un.t and 

codirg'^rr^ifbrr^TcT^^^^^^^^^ ^-d . bit ,ength. so to ma.e the 

Clock rate is periom,ed by extracting atTe ecTver s^^thf So transmission l,ne 

Rg. 7 (in case of CMI coding,. However in Rs Ss or fMr.i ! '"^ ^^PP'V 9 shown in 

in the above-ment.oned embodim::; furtLf me 3^^^^^^ -PP"'^ ^ ^^P-ate system, 
through .e selection unit 2B to .e -iplexV/ursT^^^^ 
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is srtecWJ by th. coding bi( m» adaptt.. »idlo coding unH 2 . 

^.•"rrJr.1::inSr ^oT'oTSSo,''-^ .p. an- coded .udl, d«. . 

r riTo-r x-dT^ r^- of co.p««^ o. ^ - 

andbltratodatabylhed.muH.ple.nguo.lllA'*'"' ^TO,,- 

tor .»=b m»d .r»n. >^_^;^ ^ SdT^ teL»»d InO «op.y "S"'"'- >^ 
in the embodiment of Rg. 7. the OP*'"^^ l.^^L sianal However, the content of the video data may 

each with d ^erlnt number of coding bits ^^^^^^ method from the reproduced 

^Sr^ining an optima, audio coding ^« -*^,^-:Vg«a? so^^^^^ issues an -die «t rate d^S;;^';,om 
« outnuts of the adaptive coding units 2t to 2« ana me ^'Q^ ^ o^ing the digital picture ffom 

" °^FurtSer the video coding unit 4 is compnsed ^^^'^ ^"^''^^^ variable lengths the coded v.d«. 
the Soco.;«rter 3. a variable length coding ''l'J^^^XZincoae6 video data. An interframe rate of 
H^ta and a buffer 43 for temporarily stonng the ^^^^ '^"9'" ~ ^r storing two frames' worth of the 
chSg'tdge-ent unit 8 is comprised of ^^^^^^^JZVZ comparing this with the threshold 

Rg. 8. « con-prtsos a ^J^^ ™„^ o coding b« « o. »« »<«.*" "™' -^"^ , 

r^T^,t1:Sl^^^-^^'^"^^l''-Z^^^ receiving Ihe results o, .He; 

"' ngu™ 30 sho« a conBo, elgorUbn, o. » Sd audio ^uxHon »1. ra« data 

compilson ol the rale ol cbange companson un,t 82 and g 

,n ng. 30. «rst. .e resuits of the companson ^ ^^^^ 
rate of change of the video data between frames ^^^^'^^Jj^^^^^^ tremendously, and the threshok 
w?en participants at a video conference ^'^"^ ^'^ial^^^S^oi L video and audio allocation bit ra^- 

data. Further, the frame composition of 2 bits tor vioeo 
multiplexing operation of the multiplexing unit 6. 
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bu.er43^^r^;r;*r^^^^ t^e amount data store. VB, ,n the 

than tt,e threshold value Th. are checked T^t^^ ^V'^: 1 Z7ltn T T "^'^ ^""^'^^ 

rate r.?f - - s ^izsvz ^1^^ t^::-r ^ 

S.. wh:: -nrr^;^^^^^^^^^ JZV:^e!:L. proceeds to step 

ro audio bit rate and the video brt rS a?; decWed on ^nM lV/'^"^ '""^ '^"^^ ""'^ ^^V- the 

output to the multiplexing unit 6 "it rate data Ml is 

audio" dirthrTt^re' z'^''::^j^of:^:::'Z ^^'i r ^« -^e for 

decision may also be made by iust ^n^STpe o/^^^ t^^fZ "^'^^ °' '^"^ 

me video data and audio dai^ ft exampfe" tho t^^^^^ "^r ^ "'""^ '^'^"^ *° 

frequency ratio of the audio data. " '^'"^Se of video data and the high 

ADPCM ,n rt,%'whTcl"cS?es"datr^v d'''^^ ^ ""^-9 2- The SB- 

bits of a greater nSmbe/'lTx^^pTe Ls'^e^^^^^ 'T'"^^ ^'-^^ o^^- 

» and lower order bits of a smal^ nLbe ,2 bftsf ' l A k ^"^^ * "^'S" <*«"sity 

Therefore, it is possible to round offTtal^S ^e lc^T^r^'lZT^"'^ ^ 
adaptively (6. 7. and 8 bits). As shown in Vn^ctote TrTSo Ir,^^^^ ""J""'^ *° 

converter 1 for converting the audio VnL h * ? ^' ' *® '"^smitter side is provided with an A.D 

digital sound divided into'a to^ CenTbi^p^rn S^^'Z "^^^"1^'"^ ' ~<^'"S 
for converting the video input to a dioitaf ni^\^ ! w ^ ^ra^Juency brt portion, an A/D converter 3 

variable tengm coding u:r4yL 2.rgin^2ab^ ""'V^ ^"'^'"^ ^^'^ « 

storing temporarily sad variable lenS, coded Tdi Jata , J V*"* J"^^^ ^'^^ « """^^ ^ 
audio allocation rate in accordance 2to i SiInTn^^L 5 J""^ judgement unit 44 for determining an 
for allocating a part of the vSw Sate tolf^foK I ^"'^^^ ^' ^ multiplexing unit 6 

accordance^ the allocation ~t ^ate JSd rJuSx^rr',,''* S8-ADPCM coding unit 2 in 

Further, the receiver side fs provi^ Td^ a dluSn,f '"'^ """^ "^"^ 

Signal from the transmission line ToZTj^^ Lt^^ ^^ 1/ demultiplexing the multiplexed 
converting the SB-AOPCM coded digital t^un^ r^n ^^^ T ^^'^^ ^ ^ converteM3 for 

video data, a variable length deco<«ng 1^^^^^^ TJ^LT ^ VT' temporarily storing the 

5 143. an inverse coding unit 14i for Tnverse cSina tr^ l^L"!"!"^*^^^"^*^^ "^^'^ °^ ^« ''"'^^ 

digital video signal, a 0/A convertrr l S f«r^n? I 1 * ^"^^ '^^"^^ ^''^^^ "^l^t^ issuing a 
decoding control unit le tor^ZlZ Z L^oZlZ"^ MT'' "'""^ '° ^''^^'^ ""^P*^*' « 

At the transmitter side of Rg s^th^video^n^^^^^^^^ " f ^«°'d^<=« ^.ith the allocation rate, 
subjected to filtering, then ledlf tJe coL^uni^ '^^f'^" ^'9""' 3- 

. of the amount of data, the value obLn^ mere th«n hi " '^"^ ^'^^ compression 

the variable lenoth codinn ..n.» nm/T^ „ r..'"^" <l"ant.f,ed. This quantified data is processed by 

occurrenc. ard long coles are aitoSied fa, "'iLT'", "° '"q-^ncc ol 

storM »,noora,ilv, in «,e bu/foTiau^ « « occurrenc. Th. coded d»a le 

o, >n„,^on „„e „ .e,,e;r ror o?«»^n"r:r,^X'™ 

buffer i.ag,n„n, „„„ 44 m,y '°,»" i'^'"''"' •"»' and .he 

Ihe amount ol data. "•'•"■""e ffie allocation (codin,! rate o( the audio data based on the data on 

The a^llo i„po, „ conv,„«, to digits sound by the A O convener , and sen, to the SB-AOPCM codin, 

■^o^^^.'^m'^^P^^'^^^^r", « " -ed icr selecon o- the 
.o the cutaway portion, and *e auSo ^"^""^ "J^^l "' " "-"O 

raulBplexIng unit (MUX) 6 and transmitted "Itoanon rale data are multiplexed by the 

yideo°"dr a°u* SiC? a'n^"r»2,r^%S*,,r -m^^ng unit (DMUX, , , demultiplexes the signa, to 
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,ld«,dMBlntlieMmev«»a3»lu>«n«.ng.2S>i>0"g'' ^ 

rr;jr.rrsr.r.Se^ « - - — — 

U) tha high frOTusncy componew in M BOT »uu 

Lung <5ly «» 9 bits o. a>«Jlo "'^,"^'^^1^^^^ T^. . it is iudged »a. tha amaunt ol v,*o 

to the low frequency component a totai °' ^ ^jj" ^ amount of video data ^^'^^'^^^^^^ 

® Sl>rsrtor ru«p.:Sng"^^^^^^ to - ..oca^on rate for cod.n.. w.t.o«t cutfng. 

,0 me 8 S^i^lo i^fng data from the se^>^PCM -^'^^^^ synchronization (1) is 

iSt example of the bit arrangement of FAS data is shown n r g. ^ ^ a^e muWframe 
perfo^STy'recognition of ^ '''^ «^ T^l^ 

ToC^-^S S rs^e^i-^rand 11th frames, and synch.ni.ng by the pattern 

- 34 Shows an example of the BAS ^^^^^^^ 

tt,e aiSount of the audio data and ^^^^fj^uJ^eS^ of data at the receiver side afteMhe 

transmitter side (see Rg. -"^^ " ^ ^^^^ dati demultiplexing is each one multrf^e 

frame synchronization. The units of change of P";j^^"| "l^^ g^S data is judged every 1 sub-mulbframe. 

M or^ne sub-multiframe (1 multiframe = 2 ^ub-multframes). ^e bas J y g^^dio/video data .n 

M Shown in F.g. 12. it becomes possible to recognize me d^';^'^^f^"2^P3 bAS data is recognized 

jTe next multlf^e or sub^riultiframe at the ,^Z^leor more frames match in me e.ght 

?;me malority decision at ^^^^^^'^^^^^ f ^^^^^^^ be relied on. since me BAS data a^o 

. rSsrm;%rr:e%trrauS^^^^^^ -o changes. Purmer. me demul«p.e«ng unit of 

-^ISS^^ cSe^Sn. - — IS:^ ^^^^^^^ 

^^eTti; dLTse^' to the transmission line at a rate matching me transmission speed at me pent of 
- time when me frame fomnat of Rg. 35 is com^^*^ ^^^^^ ,^„3 of the audio data 

4S Therefore, me transmission speeds according to tne vano 

in Rg. 35 are as follows: ^. _ icu/t 

\S 6 X 80 (480 bits) X 100 frames (1 second - 48 kb^s 
b 7 x 80 (560 bits) x 100 frames (1 second) - 56 kb^s 

c) 8 X 80 (640 bits) X 100 frames (1 second) = 64 kb^s construction as me 

on me omer hand, me demultiplexing un.t 1 1 ^ '^^S^^^^^^^^ direction, 
multiplexing unit 8 shown in Rg. 10 except the arrow -^^^^/^^f^^i^ation are established for .npu, 
-mat is. (1) Y.221 frame synchron.zabon and f jj"'™ the FAS data. Then, me data ,s 

data from the transmission line, as mentioned before anajV^^S demultiplexed data are sen-^ 
3:mu.tip.exed by me ^^^TT^^L lo^^o Z 88 ADFCM decoding unit 12 and me video da« 
« X:^^ - - ----^^^^^^^ wim me number of aud. 
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'^^'^'^"^^Ts^'Z^^^^ on .e a.ove ai.cca«cn .ate ... 

wherein .'^'^ fln^annel connection protccoO procedure as shown in Rg. 37. 

® The power is taimed on. then the FAS data is used for frame synchronization 

ifthferr ^AC^df^L -^'ng^?sr^;rar^•nauired about by 8 bits 

" (How"™;?'n:^^^^^^ '-^^ -^--^-ted using the BA3 data, 

coding rate.) P^c^dure is executed for communication using a fixed 

conJLlVap'pai"t^s"'^ "^^"""'^ - P^^-^.e to avoid competition with the 

" outp::rL"vidrrut?aSng%rt: v^j^r^riz'^' -^-o 

coding and decoding difference between the processing speeds of audio and video 

as audio coding unit 2 for coding the SLl s^nd 1 15? "'"'^^ing audio input to digital sound, an 
digital picture, a video codlr^S uSt 4 £ i^f ^ ^« '"put to a 

generating a video coding dSay Bme ita t^s^nc^^^^^ •!. ^'^'^^ calculation unit 3t for 

reproduced output from ^nput^St da2 of Zl^Z lLtt. "P"'<='"'=«* output and the audio 

multiplexing the coded video'data.t.l dSa, L^^^^ ^' ' -u^P'^xing unit 6 for 

sIgnSTom r ;^tS;ri^rtn" for demu,«plexingti,e multiplexed 

confrol unit 32 for delaying the audio da^ TZ:o^t:f:L''t'.T "^^'^^ ""'^ ^^^^ ^ -^^'e delay 
12 for decoding the auio data from me^l^rdeTaTSnrLTuS^ 7 ^" ^""'^ -^^'"^ unit 

decoded digital sound to an audio sianal a vidL d«X,H „l » . f ' converter 13 tor converting the 
s converterlS for conver^ng the de^oZ^^rvTderuS? tH^^^^ "^^ ^^'^ ^ 

nxedproce^'^Ini^XT.rr/rde'JS^^^^^^^^^^ obtained by subfracting the 

at the decoding unit 2 and the decod°na unT U r^L h -^"^ '"'^ processing delay Ume 

variable amount) required for the cSfn^p^ocl,^^^^^ ^^'^^ 
' time (Td: variable) required for ihe variable dSt conf?ont* « ^ H " °^ '''^ ''^'^V 
Video decoding unit U simultaneory^err L^u^^^^^^^^^^^ ^-^'"^ -d .he 

Therefore, tiie delay data Td can be found by finding Tv since Ta is rnnctJf' 

audio data (see Rg. 40). ^ multiplexed and fransmitted with the video data and 

11 ^d SvVnTthTvt^rd™^^^^^^^^^^ '^""'"^'^^^^ demu.tip,ex.ng un.t 

demultiplexed audio data by exacL the delav tim« rTl^T" '""^ '""'^"'^ ''^'^^ ^2 delays the 

in this way. adaptive delay Smrons o^rfoZn ^ '* '° ^''^ '2. 

embSnrthis^ .rTsn^^^ KoroTd^rr -'^^^ - 

(SEL) 2b for selecting the video data Vevalatton l^n^^^ TV'""^ '^^ « ^^'e-^'o^ 

video data amount detection unitlc for detcl '^T^l? T °" a 

calculation control unit 2d for receiving ttTe above ^ouT^^^^^^^ ""^ " 

select Signal, and receiving an evaluation patterrfrorr ideo c^^^^ P^«^^" 
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" «J«T,o.«.*,s,n=h™n.z«»..-n,..,.teund»=n, ^^^^ 

d«:odins^*lay_««;»- ^ ^ ?.'l„ Sl^" 1» to obtain «» delay Td and «nd to 

" « .n-bodtman. o, th. --^rSr'^L'i^ll'^'^f^ "^'^^ 

T'of hua 22 a ROM 23 Storing programs (or '"'"'^ 03. me video data, audio data. 

^Klnm>n|.b.d«ato«.^l^»^- ^ „ ,„ ^. e. th. F*S d^ l.^;-^ 

The transmission frame format is as shown '" "S- _,_curonizatlon procedure of (D frame 

r ^^^^^ ^r.s 
» ssrnr^^^^^,^ ^^^^^^^ 

.eta^CVd. ...tlonad »^ "i,"-" °' ^^1? ^^S- i ^ data .0 ba sa« .0 tt- 
data (AC data) unit shown in Rg. 43. Jnese o bi« 

- --taraiS «a opa.«on. .Ha n,u,.p,..^ 0 . ^^^JZ '^2^ 

rBrarrr„?:^«2a^^^2^^^^ 

SWt ealcylatlon unit 31 « ato«l as «» <^«""^^'^ ^'^e ,„™a, „ compMtai 
„ l*!o a, a ,a« "f^^« *™™„ " ^r'^v* .da tba «m. " - 

S'Xax^a^ IS "^"^""va^ab'. <^<'r^':Z'L data d.n,u„ip.axed b, »>. damoMplaxing un,. 

- » aa t... "dao cod,., data, .«» T.. 
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Times other than the variable delay time T of thA viw^^ r^M- 
the delay amount calculation unit ar o^tJl J^l?r?,:^**Se nZ'lT '^^''^^'^ ^" 
to prov.de a delay amount calculation unit on me receLrrdr,.2r« w 1 " Po««<ble 

In the above way. when changing me tra^sS^^^^ T 1"'^ '^'"^'^ "^^^'^^ '"^ 
20 data, the deviation becomes smaller b^ev^ so ra^^^^^^ ^ °' "''^^^ ^<=' audio 

transmission ratio of the video anrauSo d^a o™ .!^," "T"'^ ^^refore. after changing the 

corresponding to the input data so Jst^LfataZd bi^^^ * ''^'^ 

Next, an explanation will be maJorZ^ZB !°^^^^ between the picture and sound. 

sound so as to compress the silent Po.?ton iTe sSL^?aJd t^«1^.^' *^ "''^'^ "'e 
2* time. * ""^ *^smrt for a Hme con-espondlng to the delay 

termrnrffl«.rm:ro ^o^::rr^^^^^^ ^« a the «gure indicates the 

unit. 1 an A/D converter for the Lto 1^ Iud.'„ ""'"^ * ^^'^'^ '^"9"^ «'«=«'n9 

demuftiplexlng units. 9 a transmission imerfte un^ S ! k? ° " multiplexing anS 
^0 s,gnal. 141 a video decoding unit. IS a D^Atnyener tk Z l^Z^"^"^ 

system control unit. 7i a dme division coding CmrSd' rL^i^ ? ^'"^ * convener. 19 a 

The video signal is subjected to ^^TLw^^ J ""^ 'decoding unit 

subjected to variable iengmc?d?g pSce"^^^^^^^ .^^ ""'^ 

and 11. On the other hand, the auSio signSTs cTded tS^^^^^ ^ demulttplexlng units 6 

the audio coding unit 2. then the silent peX^^n SL^ii It", ! "^"^ '® '^''P" °^ ^ ''"^ ''^^ S6 kbps at 
period is extracted. This is ^sen^bled t^^'^pT^^^/^:^ ^ the signal of only the valid 

corresponding to the desired delay control iiS l te assf^hl T'T ^ P^'*^'*^ substantially 

demultiplexing units 6 and 1 1 ® assembled and supplied to the multiplexing and 

, .nterf^ ursaTJ:?pX^^^^ ^ transmission line 

signrbVrm^ir.^;^^^ .to a video signal and audio 

length decoding unit ,42. the video d^odlnVS ^T^ndZ o Tr '"^""^'^ '''^^'''^ 

video output. The audio signal passes thrcuoh til: ^ converter 15 and is received as the 

12. and the 0.A converter ?3 anS is Te^^Z'es Z Zo:TZ '"'^'^ ^^^^^'-^ 

the pe^onrs ^crxTSZuZ r"; ^^"^ °^ - - 

packet, assembles what has been assembted"n7oo2n's wnl ! T° ^'^'"^ «se'"bles it in a 

delay control amount T. and supplies Tto thrmSeZ ^ i l''"^ '""^'''^ ^ 
example. 8 kbps units in a minimum of 0 to a mLrum^^^^^^ and demultiplexing units 6 and n in. for 
unit 71 notifies the system control ^ an^ ttZZotLTln^^^ ^^Ps) Next, the time division coding 
of the audio transmission speeds of 0 kbps to 56 kbjl rhas oStpuT ''"'"""'P'^"*"^ 8 and 11 at wh,ch 

va.ue,rcor.;r^^^^^^^^^^ value (second threshold 

buffer memory in the variable fength cSg 1^42 C ° Th ''^^ '""^ ^^^^ 

increases along with the reduction in the amount of transr^ iirn ^fT^^ w -deo signal 

On the other hand, the multiolexina and rt«L , °' 
the audio Signal and descnbe L ^eTaTeVaH^^^^ ^"'^'^ "'^ transmission speed of 

ine frame data the data for notifying the opposing terminal station of the 
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corresponding te^e ^ove-meotione^^^^ ^.g^. 

period. The A/u ouvpuivs/ kwa the illustrated (A), 

packet output ®. j„„3^3ted packet data @. -s P"'^^^^^^;^ ,„Hng the time 

deiKuMplexIng units S and 1 « OO ^;2^,^„e<j audio ttansmission 3p««l « Si;^. ^ 
„ullipl»dng and ''*»r»rfd,^^rS^bW- " "iT*. cS^^T^*"" " 

°p:;'^<S'^^S^" Sifo,- , -CO.- IS P™----- - and in c«. 0. 

The audio transmission speed produced in the ^ ^ such as the threshold value ^ J" 

case of the presem wwomm 
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10 TRANSMISSION LINE 

1 1 DEMULTIPLEXINQ PART 

12 VOICE DECODING PART 

13 D/A CONVERTER 

14 VIDEO DECODING PART 

15 D/A CONVERTER 

16 DECODING CONTROL PART 

1 7 FIRST THRESHOLD VALUE 

1 8 SECOND THRESHOLD VALUE 

1 9 SYSTEM CONTROL PART 

20 TERMINAL OFFICE 

21 CPU 

22 CPU BUS 

23 ROM 

24 - 26 l/F 

27 RAM 

28 l/F 

DELAY AMOUNT CALCULATING PART 

32 VARIABLE DELAY CONTROL PART 

41 CODING PART 

42 VARIABLE LENGTH CODING PART 
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44 BUFFER DISCRIMINATING PART 
68 TRANSMISSION LINE INTERFACE 

71 TIME DIVISION CODING PART 

72 TIME DIVISION DECODING PART 
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r„il„ '°' °'"«'«"9 "W » a digital picture, 

coded „lc^. and o„tp„.:„,r:ra3°Lnri,.o^ 

transmlsslon'fremrie.^ ^ """^ Sgnal so aa ,0 obtain a co«a.n 

a-., cod', p.,... ^ 
an a„d,o deeding ^ „5, ,„ dacodlrg aaid codad sound to ot^t^n^docoded diglta, 
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4. 

5. 
6. 



decoding unit (14) based on said ^^''''.fZ^^mt^i sound to an audio signal, and 
^ .i«o »,d audio .u«p,» — fV^-i rZLrn^"" '"'^ 

transmission frame length and 

.rec««l«. urtt(« oom(»l»|^^ demuMptexlrg , co<l»l ««d. coded plctu». and 

•cor;rr^"i:iJff rf r^^-'siin^'decodad ^. 

an audio decoding unit (12) for decoding sad "°*^J°^^^btain a decoded digital picture. 

a 0/A converter (1S) for converting said decoded digital pica. 

w» 1 wheran said audio cocfing unit 
A .,d« and audio n,u,«p.o« «n.m,,sioo „^ Jf^^'i'S' « cod(n, bi. ra»s and 

said allocation signal. 

»^ ^lojrti q wherein said audio coding unit 
A and audio muMplex ™,=ml«lon s,^ ^^^^i^ZZ^ coding «. ra»s and 

iroro.t,ts;'S,~r^''»^»'-^ - 

'*rd^Sn?^:lS^ un,, ,16, pedonn, con«l baa- =" « d.mu«ip,«»d a'Sna.. ..... 

allocation bit rate. 

. A v,deo and aud. ™,«p,». ^..^...o.. ^ ™ » ^ ^ 
(2) outputs a signal of an optimal audio bit rate. 

A ..eo a. aud,o -p^^-- zi^:r:TJ^^^^ 

(5) outputs the said optimal audio bit rate as ii is as 

, A .deo and au« n,u»p«x --^ » ^ »' ^ ■ 

(2) outputs a signal of an optimal bit rate. 

„. A .deo ^ aud. .-sns.,a3,on -^^^^r ^""^ ! 

(5) outputs the said optimal audio bit rate as It IS as said aiiocauo g ^ 

1,. A^deo and audio mu.«p.ex n^^^^^^ 

(2) is an SB-ADPCM coding unit which *>"<Jes s«d ^Q-^^ «,e high frequency bit 

high frequency bit portion for coding and selects an amoum oi a. j 
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portion by said allocation signal. 

^nrtilllS'AnpS'f "^"^ ^^^'"9 ^ ^I^"" 2. Wherein said audio decodino 

^ unit performs SB-ADPCM decoding of said coded sound based on said allocation signal. ^ 

failfi SB a'SS^m S^'' transmission system according to claim 3. wherein sad audio coding unit 
(2) .s an SB-ADPCM coding unit which divides said digital sound into a low frequency bit portion Li 1 

portion by satd aJlocatiqn signal and 

Sifcatio? si^r*^"^ ^^'^^ SB-ADPCM decoding of said coded sound based on said 

faid'^Snrml^f!:' '"f T^"""''™' «^"«"9 to any one of claims 1. 3. 6. and 9 wherein 

.5 l^Ss'T^rj^J^Z S -iirl^Arr:^.^^^^^^^ <=''-9« « (8) Which 

outputs results of thelui'emeiiT "'^'^ ""'"'"'"^ ' " 

15. A video and audio rnultiplex transmission system according to claim 4. wherein said transmission unit is 

«, inSLSHa? oTLr °^ "^'"^^^ ^■""^^'"^"^ « ^«t« Of change if 

Sit^nt """P*'^ 't ^'th a threshold value, and outputs results of the 

^ilr„^l^'"° r'*'*'"^'' fransmission system according to claim 7. wherein said transmission unit is 
^te^rda^cT ' ^?"! ^ « 0' Change of 

Is faS^r'S^dtS *° transmission unit 

30 ?nt^f T^^ «S ^ r f^""* ^^''^^^^ judgement unit (8) which finds a rate of change of 

j'udge.^nt '* ' ~«P"*« the 

^J^^ ^ transmission system according to any one of claims 1. 3. 6 or 9 wherein 

said video coding unit (4) is comprised of: « ■ . o. o. or a. wnerein 

35 a video coding unit (41 ) for coding said digital picture. 

^K^^'f.S?"' ^^^^ "'^'"S '■" '«"9ths the said coded video data, 

a buffer (43) for storing temporarily said variable length coded video data, and 

da^" s3d fn ^111'°' T""""^"^ * ^"^^^ ='9"^' accordance with an amount of 

oata stored m said buffer (43) and 

4o outputs said storage amount signal. 
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(VS ^mp^sed'^ot:'""'"'''^' transmission system according ,o claim 4. wherein said video coding unit 
a video coding unit (41) for coding said digital picture 

a l^S143^7^.T"^''"''' "^'"^ """^'"^ '""9'''^ coded video data, 

a buffer (43) for stonng temporanly said variable length coded video data and 

outputs said storage amount signal. 

(V^rorJ^Jise^of:"'"'"'''*'' '^^'^"^ according ,o claim 7. wherein said video coding unit 

a video coding unit (41) for coding said digital picture 

I ITffTVlT for coding in variable lengths the said coded video data, 

a buffer (43) for stonng temporarily said variable length coded video data, and 

da?a sLed fn^^^H ^1!°' """"""'"^ ^ ^'9"^' accordance with an amount of 

oata stored in said buffer (43) and 

outputs said storage amount signal. 
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„. A Vdeo and «,dio muWp*. «»mto,=n „a«a .cccnm » ctein, ,t. ^ said vld«. codlna un« 

(4) is comprised of: 

data stored in said buffer (43) and 
outputs said storage amount signal. 
„ A v.d« and a^lo muMp^x «nam,aal<.n s,s»n, acceding „ c«n, 16. wne™« «-d =««ng un« 

(4) is comprised of: , ^ 

data stored in said buffer (43) and 
outputs said storage amount signai. 

^ A «d~ and audio n,u,«p,ox ^sn^ s,««n acceding » din. 17. .h«aU, «ld .ld~ =o<^ <^ 
9n (4) Is comprised of: . . 

25 data stored In said buffer (43) and 

outputs said storage amount signal. 

30 said interframe rate of change judgement unit (8). 

Of Change and an optimal audio bit rate, 
and optimal audio bit rate. 

of change, storage amount signal, and optimal audio bit rate. 

of change and the storage amount signal. 
. . o and audio .„«.ex .^^^^^ 

V^'-^-^^-^'^^n wNcn 9-e,a»a de,a, ..e dau (if^ 
oulpu. and audio «produc«< output Irom input and »' = 

delay Urns data is multlpteMd together by We mulliplexing umt (6). 
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said audio data In accordance with said delay time data. 

^ transmission system having a transmission unit (D) comprising: 

an A/O converter (1) for converting audio input to digital sound »/ k a 

5 a" aud'P coding unit (2) for coding said digital sound. 

an A/0 converter (3) for converting video input to a digital picture 

tJ.nT calculation unit (31) for generating delay time data for synchronizing video 

reproduced output and audio reproduced output from Input and output data of s^Jideo ^Jng S 

a^mumplexlng unit (6) for multiplexing said coded video data and audio data and said delay time 

^ ^ """"^^^^ transmission system having a reception unit (E) comprising: 

LrSS ^IHS^^ ^ a .ran:mlsJon line (10, into 

a^variable delay control unit (32) for delaying said audio data In accordance with said delay time 

an audio decoding unit (12) for decoding audio data from said variable delay control unit (32) 
a DM converter (13) for converting said decoded digital sound to an audio signal 
«o a video decoding unit (14) for decoding said coded picture, and 

a D/A converter (IS) for converting said decoded digital picture to a video signal. 

34. A video and audio multiplex transmission system having a transmission unit (D) comprising- 
an A/0 converter (1) far converting audio Input to digital sound. compnsing. 
an audio coding unit (2) for coding said digital sound, 
an A^D converter (3) for converting video input to a digital picture 

rJ^uc^^'^^^'^T generating delay time data for synchronizing video 

J^^i ^ " "^"''^ ^ f"P«t and output data of said video coding unrt 

a^rnu^plexing unit (6) for multiplexing said coded video data and audio data and said delay time 

a Teception unit (E) comprising: 

iTZS,'^::^^;^^ ^ ""^^'^-^ ^^-^ ^ mto 

a^j^anable delay control unit (32) for delaying said audio data in accordance with said delay time 

an audio decoding unit (12) for decoding audio data from said variable delay control unit (32) 
a D/A converter (13) for converting said decoded digital sound to an audio signal 
a video decoding unit (14) for decoding said coded picture, and ' 
a D/A converter (15) for converting said decoded digital picture to a video signal. 

codi; 0?;;? coded relu^r ' ^ '^'^'^ '^"^'^ ^-^'^'^ ^^"^ 

is provided, to deal with audio signals, with an audio signal coding unit (2). and 

L*SZT„n ^^H"^ T ^^'^'^ '^"9^ "*"9 (42) and the output 

ITJtT-^ f ^2) are multiplexed and transmitted, the receiver sfde 

extract the video signal and audio signal from what is transmitted 

m?fofe!trTr«nn'*° """'f'!!"' "y^^"' "eing provided with a time division coding unit 

(71) for extracting a signal dunng an effective period of a sound and forming it into a packet based 
on output from said audio coding unit (2) 

SsSotsted"'''"^ " '° " *° ' °' - -^'o 

said system control unit (19) is comprised so as to receive said audio transmission soeed and 

t:t''a^unt'i?d^i^^arr ^ ^'^^^^ -ding'^un"^^^^ 

to the amount of data in a buffer memory (43) in said variable length coding unit (42) and 
performing transmission in accordance with a frame format suited for said a'udio iansrssion speed. 
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